Evaluation of NUMA Memory Management Through Modeling and Measurements

The class of NUMA (nonuniform memory access time) shared memory architectures is becoming increasingly important with the desire for larger scale multiprocessors. In such machines, the placement and movement of code and data are crucial to performance. The operating system can play a role in managing placement through the policies and mechanisms of the virtual memory subsystem. In this paper, we explore dynamic page placement policies using two approaches that complement each other in important ways. On one hand, we measure the performance of parallel programs running on the experimental DUnX operating system kernel for the BBN GP1000 which supports a highly parameterized dynamic page placement policy. We also develop and apply an analytic model of memory system performance of a Local/Remote NUMA architecture based on approximate mean-value analysis techniques. The model assumes that a simple workload model based on a few parameters can often provide insight into the general behavior of real applications. The model is validated against experimental data obtained with DUnX while running a synthetic workload. The results of this validation show that in general, model predictions are quite good, though in some cases the model fails to include the eeect of subtle behaviors in the implementation. Experiments investigate the eeectiveness of dynamic page-placement and, in particular, dynamic multiple-copy page placement, the cost of replica-tion/coherency fault errors, and the cost of errors in deciding whether a page should move or be remotely referenced.